Update to a single large stream with multiple consumers #104
+76
−55
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
t-persson
requested review from
fredjn and
andmat900
and removed request for
a team
This feels like the more simple approach since creating a stream per testrun would require us to think a lot about security on the CreateStream endpoint. This way ETOS will handle the stream creation itself.
t-persson
force-pushed
the
single-stream-sse-server
branch
from
ebf4db1 to
7e01375
Compare
andmat900
reviewed
Mar 7, 2025
| MaxLengthBytes: stream.ByteCapacity{}.GB(2), | ||
| MaxAge: time.Second * 10, | ||
| MaxAge: time.Hour * 48, |
There was a problem hiding this comment.
Use the configuration parameter instead (test_result_timeout or similar)? I. e. max testrun duration * 2.
There was a problem hiding this comment.
Those configuration parameters are in etos library (python).
The MaxAge we set here should be controlled by RabbitMQ policies instead and those policies take precedence over our queue config: https://www.rabbitmq.com/docs/streams#retention
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Applicable Issues
Description of the Change
This feels like the more simple approach since creating a stream per testrun would require us to think a lot about security on the CreateStream endpoint. This way ETOS will handle the stream creation itself.
Alternate Designs
Possible Drawbacks
Sign-off
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
Signed-off-by: Tobias Persson [email protected]